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Abstract: Improving education quality and quality assurance (QA) have become 
the most distinctive themes of higher education (HE) reform in the 21st century. 
Although student learning outcomes (SLOs) are the most direct evidence to HE 
quality and should be an indispensable element and the starting point of education 
evaluation, in most countries evaluations have not regarded students as the most 
important target in their systems. The present paper is a case study of the Council 
for Higher Education Accreditation (CHEA) International Quality Group (CIQG) 
Quality Platform Provider (Platform) pilot review (Review) of DeTao Advanced 
Class, exploring a new paradigm for quality assurance: the Platform Review of 
SLOs in non-traditional education providers. The characteristics of the CHEA/CIQG 
Platform are as follows: 1) emphasizing the result review of actual and expected 
outcomes; 2) the product review of student learning outcomes; 3) the process 
review of student learning experiences; and 4) the development review of student 
value-added learning. 
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|. Preface 


Today in the 21st century, higher education (HE) in most countries in the world has 
entered an elite stage, quality assurance (QA) and student quality improvement 
have become the priority among priorities for the development of international 

HE. Improving education quality and assuring quality have become the most 
distinctive themes of the HE reform in today’s world. Theoretically speaking, the 
assessment of student learning outcomes (SLOs) is the most direct evidences 

of HE quality and should be an indispensable element and the starting point of 
education evaluation. However, most HE evaluation targets in most countries are 
still institutions, programs, curricula or teachers, and fail to regard the students as 
the most important target in their evaluation systems. For example, in China, higher 
education institutions (HEIs) have been exploring the new modes of producing 
high- quality graduates for ages, but they have been focused on “teaching” from the 
educators’ side, instead of “learning” from the students’ side. Educational quality is 
equal to “score,” “enrollment rate,” “administrative performance” and so on, ignoring 
students’ real educational needs. The traditional concept of “quality” resulted in 
“score first” and “standardization.” The present paper is a case study of the CHEA 
International Quality Group (CIQG) Quality Platform Provider (Platform) pilot 

review (Review) of DeTao Advanced Class. This paper explores the new paradigm 
of quality assurance: the Platform Review of SLOs to non-traditional education 
providers. 


ll. Define the Assessment of Student Learning Outcomes 


The university is not a magic box where a group of qualified graduates can 
spontaneously come out after a four-year study. “How can we recognize a person 
qualified with higher education?” “What are the expectations of SLOs?” “Are these 
expectations eventually able to be reached?” Since the birth of higher education, 
philosophers, educators, thinkers and the public have been seeking for answers 
and have been confused with these difficult problems. In the expressions of each 
country’s education policies, such HE purposes as “training whole persons,” 
“cultivating innovative talents,” “developing the people with comprehensive 
development” and others are indicated here and there. However, these statements 
are too abstract to express the specific meaning of SLOs. 


No doubt, the distinctive theme of HE quality assurance in the 21st century is 
“quality innovation.” QA agencies in the whole world are actively reforming the 
traditional evaluation methods and exploring more direct and more effective QA 
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methods. In 1979, E.W. Eisner proposed “the SLOs’ for the first time (Eisner, E.W., 
1979). The Joint Committee on Standards for Educational Evaluation (JCSEE) 
gave the definition - SLOs are the students’ expectations, i.e., SLOs are statements 
of what students should learn, know, understand and apply and/or be able to 
demonstrate after completion of a process of learning (Gullickson, Arlen R., 2003). 
The Assessment of Higher Education Learning Outcomes (AHELO) launched by 
the Organization for Economic Cooperation and Development (OECD) in 2008 is 
particularly notable, examining the SLOs of the bachelor degree recipients and its 
content to assess the students’ “general skills” and “discipline-specific skills.” As 
one of the most powerful and prominent countries in the world, the U.S.A began 

to pay more attention to SLO assessment and provided evidence to the public 

in order to show its “education quality,” since the middle of the 20th century, with 
the promotion of accountability and the public consciousness of accreditation and 
recognition in higher education. 


Throughout the whole development history of HE quality assurance, the evaluation 
cycle has been engaged in developing standards to be recognized by the public 
and finding out the effective and reliable methods to review SLOs. According to the 
statistics of Victor M.H. Borden, there are approximately 250 evaluation instruments 
(mostly examinations and surveys) (NILOA, 2015). Although today in the U.S.A., 
research on SLO assessment has greatly expanded, SLOs assessment is far 
away from the stage of professionalization and scientification. Many theoretical 
and practical topics still need to be explored. For example, the National Survey 

of Student Engagement (NSSE) and the University of California Undergraduate 
Experience Survey (UCUES) are widely applied to review SLOs and students’ 
personal development in the HEIs, which have had great academic and serial 
influence. However, criticisms have arisen: the survey statistics of learning 
outcomes are only derived from the students’ self-assessment. How can students 
accurately define their SLOs and critical thinking skills? Are students able to 
objectively report their gains of SLOs? 


And, the multiformity of different HEls, programs, majors, students, learning 
experiences, learning abilities and attitudes are quite different. Therefore, it is very 
difficult to supply an SLO definition that it can contain everything and distinguish 
among all. Peter T. Ewell pointed out that understanding the definition of SLOs 
needs the method of concept analysis method: 1) the different levels (such as 
HEls, program, student, etc.); 2) various outcomes of learning experiences (such 
as cognitive learning, career success, life satisfaction, etc.); and 3) different 
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perspectives and different observation points (such as the level after graduation 
and added value after entering university, etc.) (Ewell, Peter T., 2001). It is also 
difficult for us to give a strict distinction that can contain everything and have mutual 
definition. But, analysis methodology gives us the enlightenment by focusing on the 
core concept of SLOs, from a variety of perspectives and the relationship, we can 
get a comprehensive interpretation of SLOs’ meaning. 


The broad definition of SLOs generally refers the outcomes of “product” and 

achievements resulting from HE investments and activities, such as the number 

of graduates, social services, scientific research, learning outcomes, student 

employment, which is of universal applicability and importance and can “support” 
or “prove” the HE output of different levels. Focusing 
on the narrow definition in the present paper, SLOs 


* # 
4 4 By measuring refers to students’ comprehensive abilities achieved 
the students abilities, after completing the courses, the program and 

the degree of both the other learning activities or obtaining the degree, 


> such as expected cognition level (knowledge and 
students’ growth and 
understanding), emotion (attitude and value), 
value-added outcomes practical skills and acquisition ability. By measuring 
can be seen. ) ) the students’ abilities, the degree of both the 
students’ growth and value-added outcomes can 
be seen. Thus we have to answer at least four 
questions: 1) what knowledge will the students learn? 2) what learning experiences/ 
professional skills have the students acquired? 3) what are the students able to do? 
4) what distinguishes your students from other ones in other programs? and 5) what 
lifelong-learning ability have the students achieved? (Sz /f-#4, 2011). After having 
solved these five problems, then the extension meaning of SLOs can be clarified. 


From the perspective of organizational structure, SLO assessment can be divided 
into two aspects: internal assessment inside HEIs and external evaluation outside 
HEls. This research focuses on the latter, i.e. focusing on the practical exploration 
of CHEA/CIQG Platform Review to DeTao Advanced Class. CHEA is a national 
advocate and institutional voice for self-regulation of academic quality through 
accreditation, CHEA is an association of 3,000 degree-granting colleges and 
universities and recognizes approximately 60 institutional and programmatic 
accrediting organizations (CHEA, 2015a). One of CHEA’s main tasks is recognition 
- to recognize the qualifications and conditions of the accrediting organizations. The 
Platform Review of DeTao Advanced Class is a form of quality review - to review 
SLO basic quality of the non-traditional, innovative education providers. 
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lll. CHEA/CIQG Quality Platform Review of DeTao 
Advanced Class 


CHEA has been concerned and committed to address student achievement since 
1998. In 2012-2013, more than 23,994,000 students were enrolled in accredited 
institutions (CHEA, 2015b). Judith Eaton, CHEA President, stated “CHEA has 
engaged the issue of accreditation and student achievement in two ways. First, 
CHEA recognition of accrediting organizations addresses student achievement. 
Second, CHEA has, through a variety of publications, advisories and other efforts, 
encouraged and emphasized the importance of attention to student achievement 
in the work of accreditation” (CHEA, 2015c). In January, 2015, at the CIQG Annual 
Meeting, CHEA/CIQG Senior Advisor on International Affairs Stamenka Uvalic- 
Trumbi¢ Stamenka put forward “the Program of Quality Platform Provider’ and one 
paper titled “Higher education outside colleges and universities: how do we assure 
quality?” published as a C/QG Policy Brief in January 2014. In August 2014, CHEA/ 
CIQG held a Webinar titled “Exploring External Quality Review for Non-Institutional 
Providers.” All these activities focused on non-traditional, innovative education 
providers except the HEls. 


The Platform is designed as a response to an emerging new sector of higher 
education, offerings from private companies and other organizations, available 
alongside traditional colleges and universities. The primary intent is to assure and 
improve quality as this sector serves more and more students. The Platform is an 
outcomes-based review using standards established by the Platform, a self-review 
by the provider and peer expert review (CHEA/CIQG, 2015a). DeTao agreed to 
undertake a piloting of the Platform offerings and sent in an application to CHEA/ 
CIQG in April 2015 to become a Quality Platform Provider. 


1. Introduce DeTao Advanced Class 


DeTao is a private company set up in 2012 with the aim of developing innovative 
educational programs which go beyond conventional education approaches and 

do not belong to the traditional higher education system in China. The programs 
are designed and implemented with the guidance of world-class Masters with 
distinguished academic or industry backgrounds in a variety of disciplines. The 
educational branch of DeTao, DeTao Masters Heritage (DMH), has developed three 
major educational programs: Advanced Class, Industrial Training, and O+O (Online 
and Onsite) Learning. The Review is focused on the Advanced Class that may be 
used toward a degree conferred by the Shanghai Institute of Visual Arts (SIVA) to 
selected students. 
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Advanced Class is aimed at providing high-quality bachelor-level educational 
content to Chinese universities. It provides the students a chance to receive 
education as if they were abroad. All the teaching teams are selected in a strict 
way. Advanced Class officially kicked off the recruitment from 2013. The first two 
majors are Strategic Design and Innovation (SDI), and Creative Animation (IACC). 
As of September 2015, the total number of enrolled students in 10 majors with 13 
Masters is 457 (see table 1). 


Table 1: List of Basic Information of DeTao Advanced Class 


Year of Faculty 
Entry No. 
1 Hartmut Esslinger | Product Design (Strategic aaa ie 2014 | 48 (20+19+9) | 35 
Design and Innovation) 
ai Wynants Product Design (Sustainable | 2014 & 2015 | 42 (19+23) 
Furniture Design) 


pe Dotan Environment Design 2014 & 2015 | 48 (22+26) 
(Ecological Architecture 
Design) 
Tina Hart & Kim Environment Design 2014& 2015 | 51 (24+27) 20 
Jarrett (Themed Environmental 
Design) 
Wang Min & Visual Communication 2014 & 2015 | 48 (22+26) 23 
Michel de Boer Design (Branding, Identity 
and Public Space) 


Josep Henriquez Performance (Spanish 2014 & 2015 | 9 (6+3) 
Classical Guitar) 
7 Robin King Animation (Creative 2013 & 2014 | 74 (29+27+18) | 13 
Animation) & 2015 
Pe Ascott Art & Technology 2014 & 2015 | 40 (18+22) 
(Technoetic Arts) 


ambled Baeriswyl Cultural Industry 2014 & 2015 | 44 (20+24) 
Management (Brand 
Strategy and Management) 
Patrick Gottelier & | Fashion & Apparel Design 2014 & 2015 | 53 (24+29) 
Jane Gottelier (Fashion: Knitwear’ 
—— Design) 


Ea a Ca 


*Note: 1) Number of faculty is calculated by summing up masters, DeTao teachers, SIVA teachers and visiting 
experts; 2) The total number of Advanced Class faculty is calculated by summing up 10 classes, eliminating the 
repetition. 
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The overall cultivation target of Advanced Class is to help Chinese universities 
create highly ranked international subjects and to cultivate innovative, 
comprehensive graduates for all industries. The courses are designed and 

lead by the international top industrial masters and professors. The program is 
project-based and focuses on training the students’ hands-on skills and project 
management skills. After four years, the students will receive bachelors degrees 
granted by the partner universities and certificates with the signature of the Masters 
issued by DeTao. The students gain knowledge and skill and have great potential to 
be the industrial elites. 


2. The Quality Platform Review Preparation 


With the coordination of DeTao Masters Stamenka Uvali¢-Trumbié and Sir John 
Daniel, former Chair of the CIQG Advisory Council, on 2-3 June, 2015, Judith Eaton 
and other experts held “Training Workshop of DeTao Self-evaluation Using the 
CIQG Quality Platform Standards” for DeTao in China. It included the following four 
presentations: “Context of the workshop within DeTao’s development: expected 
outcomes and follow-up” by Sir John Daniel and Stamenka Uvalié-Trumbic¢, “Global 
trends in quality assurance and accreditation and context of the Quality Platform’ by 
Judith Eaton, “Purposes of the self-review and presentation of the four standards” 
by Dorte Kristofferson, “Purposes of the self-review and presentation of the four 
standards linking them to the Chinese context’ by Jianxin Zhang, as well as making 
SER frameworks by the four participant groups, etc. CHEA/CIQG review focuses on 
solving four problems: 1) why to review? (aim); 2) what to review? (content); 3) who 
to review? (bodies); and 4) how to review? (methods) (see fig.1). 


Fig. 1: Basic Elements of CHEA/CIQG Platform Review of the SLOs 


Review Contents ~ 


1. SLOs 
2. Expected outcomes 
3. Credit transfer 
4. Transparency & 
comparability 


Review Aim 


1. Review & judge 
2. Complete & develop 


Review Bodies 
Review Methods 


Combination of SER & actual 
review; quantitative & 
qualitative evaluation method 


1. Internal QA party 
2. External QAA 
3. Stakeholders 
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The workshop had four purposes: 1) to remind DeTao staff and Masters briefly 
about the basics of quality assurance; 2) to provide training on how to use the 
Quality Platform standards for a self-review of DeTao’s educational and executive 
development programs; 3) to allow the CHEA team to look at DeTao’s existing 
documentation about learning objectives for the advanced courses; and 4) to 
prepare for the next step, an external review, leading to acceptance of DeTao as 
a CHEA Quality Platform Provider (Uvali¢-Trumbi¢, Stamenka & Daniel, Sir John, 
2015). The two-day workshop achieved a satisfactory effect for both the experts 
and the trainees from DeTao, which can be said that it is a good case of the 
cooperation between internal QA and external QA. 


3. The Quality Platform Review Standards 


Advancing the understanding of international quality issues is essential to promote 
high-quality HE in today’s competitive and internationalized world. CIQG meets 
this need “serving as a valuable forum for colleges, universities, accrediting and 
quality assurance bodies and others worldwide to address issues, challenges and 
opportunities, all focused on academic 
quality’ (CIQG, 2012). Based on the 
te above mission, CHEA/CIQG has 
adieed Advancing the developed four standards of the Platform 
understanding of Review of the SLOs (see Table 2). 
international quality issues 


is essential to promote high- The Platform is an assessment of 


sustainable development based on the 


quality higher education “evidence,” emphasizing “providing the 
in today’s competitive and evidence of SLOs,” i.e., through the 
internationalized world. , -} providers’ long-term accumulation of 


education process and scientific collection 

of statistics and obvious evidences, by 

adopting the method of “the combination 
of SER and actual review as well as quantitative and qualitative evaluation,” the 
review experts comprehensively and systematically “review and judge” the overall 
situation of the provider’s SLOs and put forward constructive suggestions for its 
“future completion and development.” 
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Table 2: The Four Standards of the Platform Review of SLOs 


Four Standards [Description __—~=drEvidence——SSSSSSCSC~=*Y 


1. Learning 
Outcomes are 
Articulated and 
Achieved. 


2.SLOs Meet 
Postsecondary- 
level Learning 
Expectations. 


3. Curricula Provide 
an Opportunity 
for Successful 


Transfer of Credit. 


4. Transparency is 
Maintained and 
Comparability is 
Established. 


The provider organizes its work, 
determines the content of offerings 
and sets expectations of rigor based 
on anticipated and actual Outcomes 
for students: information about gain in 
skills, competencies or other attributes 
resulting from a learning experience. 


The provider demonstrates that the 
articulated and achieved SLOs are 
consistent with expectations of student 
learning at degree-granting colleges 
and universities. 


For the provider's offerings intended 

to be used for credit or credentialing 

at a college or university, the provider: 
1) Builds opportunity for student 
progression beyond its offerings as part 
of its curriculum development; 

2) Organizes offerings into a coherent 
learning experience that can be 
sustained across multiple providers of 
higher education. 


The provider develops and provides 
reliable, easily accessible and readily 
understandable information to the 
public, at least annually, about 

its performance: 1) An aggregate 
description of the SLOs that are 
achieved; 2) The Outcomes of 
comparisons of performance among 
similar types of non-institutional 
providers; 3) An aggregate description 
of the uses of the offerings to students, 
for example, advancing toward an 
educational goal, employment. 
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e Expectations of SLOs have been 


developed and are available for all 
students and across all offerings. 


Documentation of student learning 
gains, competencies and other 
attributes as identified is provided 
(omitted). 


Description of the basis on which the 
organization judges the performance 
of faculty, the content of curriculum 
and the progress of students is 
provided. 


Description of the basis used to 
determine whether outcomes are 
to be considered as postsecondary 
is available. This description may 
include, for example, comparison 
with offerings of other providers of 
postsecondary learning. 


Description and documentation is 
provided of opportunities for students 
to successfully use the offerings as 
part of meeting broader education 
goals. 


Material is provided about a context 
for the offerings in relation to 
generally accepted curricular content 
throughout higher education. 


Documentation of student 
achievement from the provider and 
other similar providers is available. 


Information is routinely provided 

to students and the public about 
institutional performance in terms of 
attainment of SLOs, either individual 
or in the aggregate. 
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4. The Quality Platform Review Process 


CHEA/CIQG Platform Review of SLOs consists of the following six stages: 1) 
review application; 2) self-review (SR) workshop; 3) SR report; 4) desk review; 5) 
site visit; and 6) review result (see Fig 2). 


Fig. 2: Flow Chart Showing the Process of CHEA/CIQG Platform Review of the SLOs 


Stage 1: Review Application 


DeTao is seeking a professional quality reviewer to be reviewed. In late April 2015, 
DeTao submitted its application to CHEA/CIQG. 


Stage 2: Self-Review (SR) Workshop 


On 2-3 June, 2015, an expert group made up of Judith Eaton and other four experts 
held “Training workshop of DeTao Self-evaluation using the CIQG Quality Platform 
Standards” in DeTao in China in order to assist DeTao to prepare its self-review and 
be ready for site visit. The experts had in-depth interviews to DeTao staff and visited 
DeTao infrastructure such as fashion design studio, animation studio, architectural 
design studio, green building studio and other locations. Both the experts and 
DeTao staff had a clearer understanding on the review of the SLOs. 


Stage 3: SR Report 


From late June to mid-September, DeTao formed the self-review (SE) team and 
began its SR according to CHEA/CIQG review standards. The self-review report 
(SER) is made up of three sections: 1) provider information; 2) evidence that quality 
platform standards are met and 3) two annexes - examples of evidence as needed. 
After three months’ work, DeTao submitted the SER with 26 annexes as supporting 
evidence. 
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Stage 4: Desk Review 


From September to the end of October, the review panel of international experts 
did a thorough desk review to DeTao SER. Each of them made the individual 
preliminary review judgment and made a list of unknown questions according to the 
analysis to the SER. 


Two general types of assessment methods have been adopted. One is a direct 
method (quantitative method, primary data): 1) demonstration of expected SLOs; 
2) providing evidence of SLOs; and 3) actual samples of students work, etc. The 
other is an indirect method (qualitative method, supplemental data): 1) students, 
staff or others report their perception of how well a given learning outcome has 
been achieved; 2) opinions or thoughts about student learning (not based directly 
on student performance); and 3) gathering information through means other than 
actual samples of students work, such as employers surveys, quality assurance 
agencies’ reviews, case studies of cohort groups, etc. 


Stage 5: Site Visit 


On 3-5 November, 2015, the site visit was carried out by a panel of international 
experts selected by CHEA/CIQG with the aim of making a judgment whether DeTao 
Advanced Class meets the Quality Platform standards. The coordinators were 
CHEA’s President and CHEA’s Senior Advisor on International Affairs, the panel 
leader was Dorte Kristoffersen, executive director 
of Hong Kong Council for Accreditation of Academic 
and Vocational Qualifications (HKCAAVQ), and the 


* + 
ee sates ~~ The aim [was 
experts were Axel Aerden, senior internationalization [ ] 


policy advisor of Accreditation Organization of the making a judgment 
Netherlands and Flanders (NVAO) and Jianxin whether DeTao 
Zhang, chief expert of Yunnan Higher Education Advanced Class meets 
Evaluation Centre (YHEEC) in China. Three main We Quality Platform 


methods were adopted in the site visit: 1) visits to 
the SLO exhibition introduced by the Masters and standards. .}. | 

to observe the student practice in Masters studios; 

2) focus groups: the review panel held five focus 

groups made up of five categories of staff -a DeTao SR group members, Masters, 
coordinators and teachers, student representatives, educational administrators; and 
3) one-to-one depth interview with cooperative party and held an Internet remote 
interview with the third party from the enterprise (see Table 3). 
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Table 3: The Site Visit Program (shorten) 
Time Interviewees Place 
Visit Master Studio on the Basement Basement 
9:30 — 10:20 
10:30 ~12:00 


13:35 —14:35 | Interview seven coordinators and teachers 


14:40 —15:30 | Interview one leader from SIVA (cooperative institution) 


7F-P1 


Interview one stakeholder (enterprise representatives) 7F-P2 


15:40 -16:30 | Interview eight students from four majors 
16:40 —-17:30 | Interview two teaching administrators 


On the basis of the desk review to DeTao SER, the site visit has three purposes: 

1) to affirm the information in the self-review report that the provider meets the 
Quality Platform standards; 2) to obtain any needed additional information or 
responses to questions that have emerged from examination of the SER; and 

3) to judge about whether the provider meets the standards (CIQG, 2015). The 
main content of the site review is carried on based on the interview outline, the 
questions from desk review and DeTao SER, the focus is to investigate and verify all 
the information involved in the SER. 


Stage 6: Review Result 


From early November to mid-December, 2015, the review panel and coordinators 
from CHEA/CIQG discussed the review results. On December 15, CHEA/CIQG 
submitted the review results to DeTao. “The Quality Platform Provider Pilot Review 
Report on DeTao Advanced Classes” is made up of four parts: 1) background; 

2) DeTao and the Quality Platform standards; 3) other comments; and 4) two 
appendixes: review panel and site visit programs. In the “DeTao Transmittal Letter 
Signed,” CHEA President says, “based on the self-review documentation submitted 
in September 2015 and the site visit conducted in November 2015, | am pleased to 
inform you that the panel is recommending that DeTao Advanced Class has met all 
requirements and standards to become a Quality Platform Provider 2016-2019 and 
CHEA/CIQG has accepted this recommendation. The review panel of Chinese and 
international experts are most complimentary about the Advanced Classes and the 
fine work that you are doing. The documentation you provided was excellent and 
the site visit was most informative. The panel has also offered several suggestions 
for ongoing improvement’ (CHEA President, 2015). 
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The program of CHEA/CIQG Platform Review is designed with high level of 
professionalism, the entire review was carried out in an orderly manner. What is 
more, from the very beginning of the SR workshop, the experts have repeatedly 
stressed, review is not only the review result of judgment and recognition, but also 
is a process of stressing DeTao provider’s summing-up, introspection, improvement 
and development. 


IV. The Characteristics of the CHEA/CIQG Quality Platform 
Review of SLOs 


As early as 1967, the famous American evaluation expert D. L. Stufflebeam 
proposed the famous “CIPP” (Context, Input, Process and Product) evaluation 
model. It is famous for its remarkable characteristics: product, process and 
feedback, etc., which have been widely used in many kinds of education 
evaluations. We consider CHEA/CIQG Platform Review of SLOs has adopted some 
characteristics of the CIPP evaluation model, and made further steps, the standards 
of the Platform (see Table 2 above) is made up of “OPPTTC’”, i.e. outcome, product, 
process, transformation, transparency and comparability (see Table 4). 


Table 4: The “OPPTTC” of CHEA/CIQG Platform Review of SLOs 


Key Element | Description of OPPTTC 


Outcome To review the matching between actual and expected outcomes, verifying 
the degree of consistency of SLOs’ expected goals and actual goals, i.e. the 
educational goal-referenced outcomes 


Product To review the product of students’ ability improvement and their personal 
development, verifying the education provider’s “product” quality 


Process To review the whole process of students’ learning, curriculum implementation, 
SLOs, verifying the changes before, during and after students’ learning behaviors 


Transformation | To review the transformation of student credits and sustainable development 
of the students’ learning behaviors, verifying whether SLOs can be successfully 
applied in other similar providers 


Transparency | To review whether the provider develops and provides reliable, easily accessible 
and readily understandable information to the public, verifying the process 
openness of the providers 


Comparability | To review the comparison of the quality and importance of SLOs with other similar 
providers, verifying the international standards and procedures 
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The CHEA/CIQG Platform Review not only proves a good quality platform to a 
qualified provider, but also promotes the provider’s sustainable development, 
highlighting the quality of “OPPTTC” model. The most prominent feature is the 
following 4 aspects: 1) the actual and expected outcomes; 2) the product of learning 
effect; 3) the process of learning experience; and 4) the development of value- 
added learning (including transformation and comparability). 


1. Emphasize the Result Review of Actual and Expected Outcomes 


From the perspective of the process of training talent, SLOs can be divided into two 
types: one is actual learning outcomes, i.e., what students have achieved in the 
learning process; the other is expected learning outcomes, i.e. the provider expects 
the students have achieved the education and curriculum goals after the learning 
process happens. Thus, the SLOs review contains two aspects: 1) actual outcomes 
of SLOs, i.e., effect outcomes, which measure and review students’ practical 
learning outcomes to prove the quality and importance that curriculum and program 
have produced to the students; 2) expected outcomes, i.e., goal-referenced 
outcomes, which measure and judge the degree of consistency of SLOs’ expected 
goals and actual goals after a period of learning. Comparatively, expected SLOs 
are worth special attention because it has the following three features: 1) to meet 
the enterprises’ needs to graduates’ knowledge and ability; 2) to meet the needs of 
course teaching content to students’ training goals; and 3) to meet students’ needs 
to their knowledge, skills and abilities. 


The Platform has two SLOs definitions of the above two meanings: 1) the provider 
must supply what students know (cognition), think (attitude) and do (behavior) 
through actual performance or specific behavior of each student himself/herself, 
team or class in the process of students’ training or course teaching; 2) the provider 
must supply evidence to accurately express SLOs in the guidance of the education 
teaching and curriculum goals. In the CHEA/CIQG review, the focus interview is 

on the combination of actual and expected outcomes, based on nine elements of a 
complete curriculum syllables: 1) survey report; 2) executive summary; 3) objective; 
4) four-year training plan; 5) course description and assessment; 6) course outline; 
7) student selection; 8) grading system; and 9) career prospects (Shanghai DeTao, 
2015). The experts asked the relationship between the thirteen courses’ objectives 
and the general objectives of DeTao education. Does the objective of each course 
must match DeTao mandate of “congregating world-class masters, collecting 
industrial wisdom, nurturing professional elites, and fueling corporate development” 
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(Shanghai DeTao, 2015)? We can say that actual SLOs are the reflection and 
carrier of HE quality, expected SLOs is the concrete embodiment of the educational 
objectives, and the SLOs is the link between education objectives and education 
quality. 


2. Emphasize the Product Review of Student Learning Outcomes 


As stated in the preface, most indicators and standards are mainly on HEls’ basic 
infrastructure, programs, curricula, teachers, libraries and other input items while 
paying little attention to the students, let alone SLOs. This is just like in kitchen the 
chef’s qualifications (like teachers), cooking materials and utensils (like curriculum 
and infrastructure) are evaluated, but not actual dishes (like student)” (248 4H, 
2008). To a certain extent, “learning” effect is from “teaching,” but “teaching” effect 
is not necessarily a reflection of “learning” effect. 


As a “factory” of educational output, the provider is responsible for the quality 

of its “products.” The Platform Review of the SLOs has changed from the 
educational “input” to “output.” For example, “documentation of student learning 
gains, competencies and other attributes as identified” and “demonstrates that the 
articulated and achieved student learning outcomes...” (CHEA/CIQG, 2015b ) can 
be given: 1) analysis of student transcripts, teaching syllabi and curricula contents; 
2) student performance, exhibitions and simulations; 3) observations of student 
behaviors, learning attitudes, values and experiences (including internships); 4) 
student self-reviews on their own skills, abilities and progress; 5) student landmark 
works of experiments and practices; 6) graduation theses or research projects; 7) 
student portfolios of learning experiences; 8) in-depth interviews among students 
and teachers; 9) tracking data after graduating, further studies, employment; 

and 10) feedbacks of alumni, enterprises and employers, etc. These methods 

can be used to review students’ “general knowledge and skills,” and also be 

used to review their “discipline-specific knowledge and skills” like that of OECD 
mentioned above. During desk review, the experts pay special attention to students 
“product,” especially to the exhibitions showed in DeTao’s SER along with the 

26 appendixes. In “the Proposal of Proposal for the Development of a Standards 
System for Chinese Animation Education and Training,” Master Robin King from 
Major Animation not only stresses the national and international industry validation 
of competency, but also concerns for students’ core competency for animation 
expertise in a “performance grid” (King, Robin, 2015), which is commended by the 
panel expert. 
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For the concepts and methods of the QA development, the Platform review of 
SLOs was an extraordinary contribution: It has shifted from the “input evaluation” to 
“output evaluation,” which is a revolution to QA. From the perspective of a steering 
baton, the SLOs assessment will shake the long-standing traditional teaching 
methods and promoting the shift from teacher-and-teaching-centered traditional 
paradigm to student-and-learning-centered modern paradigm. 


3. Emphasize the Process Review of Student Learning Experiences 


It can be said that the assessment is to review the specific student learning 
procedure of SLOs. As an operational concept, the process of SLOs is as follows: 
before student learning behaviors, the provider establishes the student training 
programs according to the expected goals of the public; during the learning 
experience, the teachers, administrators and other staff transmit knowledge in 
teaching and experimental practices and other academic activities; after the 
learning experience, students show the knowledge, skills and abilities that they 
apply some kinds of outcomes, namely “make — transmit - achieve - apply” process 
(see Fig. 3). 


Figure 3 The Process of the SLOs before, during and after the Learning Behaviors 
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The Platform Review of SLOs stresses that SLOs’ content is dynamic, including the 
“expectation - implementation - assessment” process of all the activities, before, 
during and after the student learning. The experts observed that the thirteen 
Advanced Classes have their own different curriculum types according to their 
curriculum syllabi. For example, “Strategic Design and Innovation” uses “project- 
run-through pattern,” i.e., based on the project-based learning, the students 
synchronized to complete six main courses, and in four school years, six main 
courses will gradually develop and become a progressive development. 


In the review feedback, the experts recommend that “to introduce the mechanism 
for the students’ portfolio assessment both in soft and hard copy with an 
emphasis on the whole student learning process” (Review Panel, 2015). “Portfolio 
assessment” is a new type of qualitative assessment tool which can be used 

to objectively and comprehensively evaluate students. Portfolio includes the 
whole process of student learning: 1) evidence of the course projects, best work 
and progress in the school year; 2) learning outcomes of social practices and 
experiments and other activities outside the course; 3) evidence of student growth 
and change (skills, interests, attitudes); 4) comments from peers, teachers, 
enterprises and other stakeholders; and 5) evidence of student self-reflection, self- 
cognition and self-appraisal. 


4. Emphasize the Development Review of Student Value-added 
Learning 


In 1979, E.W. Eisner proposed the concept of “learning outcome” to emphasize 
the “added value” of student learning. Since the middle of the 1980s, Terry Taylor, 
Charles McClain and other experts put forward the value-added evaluation method. 
By analyzing students’ learning process and outcomes during the university 
years, the added value or progress of student learning can be articulated and 
achieved. The added value can be regarded as the outcome of the improvement 
of teaching quality, the symbol of HE development (3 44, 2007). Pay attention 
to the value-added increment of students’ “before-during-after” learning activities, 
i.e. by analyzing the SLOs after a certain stage of learning process, then we know 
what additional value have the students get. This increment can be considered 

as the result of the improvement of teaching quality, which is also the focus of QA 
evaluation. 


Since DeTao has only one partner, SIVA, the experts recommended that DeTao 
“strengthen cooperation with degree-granting institutions in China and overseas 
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in order to ensure adequate pathways for students as well as opportunities for 
benchmarking with comparable institutions’(Review Panel, 2015). This is also one 
of the measures to ensure that SLOs have added value, which is used to express 
the students’ development of knowledge, skills, and ability. The Platform framework 
emphasizes the following four aspects to added value: 1) ability of learning 
knowledge; 2) ability of critical thinking and innovation; 3) ability of professional 
skills; and 4) hands-on ability. 


No doubt, sufficient value-added evidences of SLOs are the guiding ideology and 
logical starting point for QA. However, because there are no graduates in DeTao 
Advanced Class, the discussion on added value will be in the near future. However, 
the QA mode of the Platform review of SLOs confronting the students’ value-added 
learning and emphasizes the outcome evidences will become the QA trend of 
higher education. 


V. Concluding Thoughts 


By the end of 2015, the CHEA/CIQG Platform Review of SLOs to innovative and 
non-traditional providers has been completed. The assessment of SLOs can be 
considered both as an end and a means. It is not a one-time event, but a dynamic 
on-going process, a process of systematic collection and analysis of SLOs to 
improve student learning. It is a conclusion, but also a beginning. There are still 
many problems need to be further explored: How to strengthen the alignment 
between the heading and the explanatory statement of the quality standards and to 
consider their clarity for non-native speakers? How to promote the establishment of 
the assessment system of the formation evaluation of the provider in order to pass 
the re-review after three years? How to form a suitable definition of “SLOs’” fitting 
both the provider and the review panel? 


Modern education has surpassed the traditional “autonomy” and entered a new 

era of “quality governance.” The real meaning of the label of the Platform is the 
need for the sustainable concern for external and internal QAAs as well as related 
stakeholders to establish a more effective system for non-traditional, innovative 
education providers, to provide student learning experience with good quality, to 
provide better quality education services for the public and the cooperative HEls. 
At the same time, it can help CHEA/CIQG improve review system of the Platform to 
service the providers with good quality and supply the international education with 
more QA experience. 
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